Binary Neural Networks Algorithms, Architectures, and Applications (Baochang Zhang, Sheng Xu, Mingbao Lin etc.)

Algorithms for Binary Neural Networks

D(SRFKZLVHRVFLOODWLRQUDWLR

E(SRFKZLVHWUDLQLQJORVV

FIGURE 3.30

(a) Epoch-wise weight oscillation ratio of ReActNet (solid), ReCU (dotted), and ReBNN

(dashed). (b) Comparing the loss curves of ReActNet and our ReBNN with diﬀerent calcu-

lations of γ.

of ResNet-18. As shown in Fig. 3.30(a), the dashed lines gain much lower magnitudes than

the solid (ReActNet) and dotted (ReCU [267]) lines with the same color, validating the

eﬀectiveness of our ReBNN in suppressing the consecutive weight oscillation. Besides, the

sequential weight oscillation ratios of ReBNN are gradually decreased to 0 as the training

converges.